This code imports, cleans, and merges MODIS product data exported using the AppEEARS web application.

Exported MODIS data for the period 2000-2016:

Data was exported for 50 BAMS paper sites (.csv format, “ID”, “Category”, “LAT”, “LONG”)


Attaching package: <U+393C><U+3E31>readxl<U+393C><U+3E32>

The following object is masked from <U+393C><U+3E31>package:officer<U+393C><U+3E32>:

    read_xlsx

data.table 1.12.0  Latest news: r-datatable.com

Attaching package: <U+393C><U+3E31>data.table<U+393C><U+3E32>

The following objects are masked from <U+393C><U+3E31>package:dplyr<U+393C><U+3E32>:

    between, first, last

The following object is masked from <U+393C><U+3E31>package:purrr<U+393C><U+3E32>:

    transpose

The following objects are masked from <U+393C><U+3E31>package:lubridate<U+393C><U+3E32>:

    hour, isoweek, mday, minute, month, quarter, second, wday, week, yday, year


Attaching package: <U+393C><U+3E31>magrittr<U+393C><U+3E32>

The following object is masked from <U+393C><U+3E31>package:purrr<U+393C><U+3E32>:

    set_names

The following object is masked from <U+393C><U+3E31>package:tidyr<U+393C><U+3E32>:

    extract

9 MODIS products are downloaded in 5 .csv files:

[1] "./CH4-V2-0-MOD09A1-006-results.csv"  "./CH4-V2-0-MOD11A2-006-results.csv"  "./CH4-V2-0-MOD13Q1-006-results.csv" 
[4] "./CH4-V2-0-MOD15A2H-006-results.csv"
[[1]]

[[2]]

[[3]]

[[4]]
NA

Simplify data by removing unnecssary fields:

$vis

$lst_day

$evi

$lai

$lst_night
NA

Look at what the data quality levels are, convert good to 1, keep only 1s

$vis

$lst_day

$evi

$lai

$lst_night
NA

Make a plot of daytime LST to make sure things look OK

Create a dataframe for each product consisting of LAT, LONG, DATE and the product field (e.g. LSTD)

Create a full dataframe with each site name/latlong repeated n = length(2001-01-01 to 2014-12-31) (FLUXCOM data window). Then right_join each modis product to the site name/all dates tibble to produce a complete dataframe of modis data.

full.date <- as_tibble(rep(as_date(c(as_date("2001-01-01"):as_date("2014-12-31"))),59)) ## rep by number of sites 
names(full.date) <- "DATE"
full.date <- full.date %>% add_column(ID = rep(bams.sites$ID, each = length(full.date$DATE)/59)) %>% 
                              add_column(LAT = rep(bams.sites$LAT, each = length(full.date$DATE)/59)) %>%
                                add_column(LONG = rep(bams.sites$LONG, each = length(full.date$DATE)/59))
modis.full <- full_join(full.date, lst_d.filtered, by =c("DATE","LAT","LONG"))
modis.full <- full_join(modis.full, lst_n.filtered, by=c("DATE","LAT","LONG"))
modis.full <- full_join(modis.full, evi.filtered, by=c("DATE","LAT","LONG"))
modis.full <- full_join(modis.full, b02.filtered, by = c("DATE",'LAT',"LONG"))
modis.full <- full_join(modis.full, b05.filtered, by = c("DATE","LAT","LONG"))
modis.full <- full_join(modis.full, lai.filtered, by=c("DATE","LAT","LONG"))
modis.full <- modis.full %>% distinct(ID, DATE, .keep_all = TRUE)
str(modis.full)
Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   301669 obs. of  10 variables:
 $ DATE: Date, format: "2001-01-01" "2001-01-02" "2001-01-03" "2001-01-04" ...
 $ ID  : Factor w/ 59 levels "ATNeu.csv","BCBog.csv",..: 1 1 1 1 1 1 1 1 1 1 ...
 $ LAT : num  47.1 47.1 47.1 47.1 47.1 ...
 $ LONG: num  11.3 11.3 11.3 11.3 11.3 ...
 $ LSTD: num  NA NA NA NA NA NA NA NA NA NA ...
 $ LSTN: num  NA NA NA NA NA ...
 $ EVI : num  NA NA NA NA NA NA NA NA NA NA ...
 $ b02 : num  0.198 NA NA NA NA ...
 $ b05 : num  0.169 NA NA NA NA ...
 $ LAI : num  NA NA NA NA NA NA NA NA NA NA ...

Finished! Save as .csv

# set wd
setwd("C:/Users/gavin/Box/MODIS Data/AppEEARS Data")
write.csv(modis.full, "V2.0_MODIS.csv")
LS0tDQp0aXRsZTogIlByZXBhcmUgTU9ESVMgRGF0YSBWMi4wIg0Kb3V0cHV0OiBodG1sX25vdGVib29rDQotLS0NCg0KVGhpcyBjb2RlIGltcG9ydHMsIGNsZWFucywgYW5kIG1lcmdlcyBNT0RJUyBwcm9kdWN0IGRhdGEgZXhwb3J0ZWQgdXNpbmcgdGhlIFtBcHBFRUFSU10oaHR0cHM6Ly9scGRhYWNzdmMuY3IudXNncy5nb3YvYXBwZWVhcnMpIHdlYiBhcHBsaWNhdGlvbi4NCg0KRXhwb3J0ZWQgTU9ESVMgZGF0YSBmb3IgdGhlIHBlcmlvZCAyMDAwLTIwMTY6DQoNCiogKipNT0QwOUExOioqIHN1cmYgcmVmbGVjdGFuY2UgYjAyIGIwNQ0KKiAqKk1PRDExQTI6KiogTFNUIERheSAxa20gJiBMU1QgTmlnaHQgMWttICg4IGRheSkNCiogKipNT0QxM1ExOioqIEVWSSAyNTBtICgxNiBkYXkpDQoqICoqTU9EMTVBMkg6KiogZlBBUiA1MDBtICYgTEFJIDUwMG0gKDggZGF5KQ0KDQpEYXRhIHdhcyBleHBvcnRlZCBmb3IgNTAgQkFNUyBwYXBlciBzaXRlcyAoLmNzdiBmb3JtYXQsICJJRCIsICJDYXRlZ29yeSIsICJMQVQiLCAiTE9ORyIpDQpgYGB7ciBlY2hvPUZBTFNFLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQ0KIyBjbGVhciB3b3Jrc3BhY2UgYW5kIGdldCB3ZA0Kcm0obGlzdD1scygpKQ0KIyBjaGFuZ2Ugd2QNCnNldHdkKCJDOi9Vc2Vycy9nYXZpbi9Cb3gvTU9ESVMgRGF0YS9BcHBFRUFSUyBEYXRhLyIpDQpiYW1zLnNpdGVzIDwtIHJlYWQuY3N2KCJWMi4wX3NpdGVzLmNzdiIpDQpuYW1lcyhiYW1zLnNpdGVzKVtjKDMsNCldIDwtIGMoIkxBVCIsICJMT05HIikNCmJhbXMuc2l0ZXMNCmBgYA0KDQoNCmBgYHtyIGVjaG89RkFMU0V9DQojIGxpYnJhcmllcw0KIyBsb2FkIHRpZHl2ZXJzZSBsaWJyYXJpZXMNCmxpYnJhcnkoZHBseXIpDQpsaWJyYXJ5KHRpZHlyKQ0KbGlicmFyeShyZWFkcikNCmxpYnJhcnkocmVhZHhsKQ0KbGlicmFyeShzdHJpbmdyKQ0KbGlicmFyeShsdWJyaWRhdGUpDQpsaWJyYXJ5KFJDb2xvckJyZXdlcikNCmxpYnJhcnkoZ2dwbG90MikNCmxpYnJhcnkoZGF0YS50YWJsZSkNCmxpYnJhcnkobWFncml0dHIpDQpsaWJyYXJ5KGNhcmV0KQ0KbGlicmFyeShjYVRvb2xzKQ0KbGlicmFyeShyYW5nZXIpDQpsaWJyYXJ5KHRpYmJsZSkNCmBgYA0KDQo5IE1PRElTIHByb2R1Y3RzIGFyZSBkb3dubG9hZGVkIGluIDUgLmNzdiBmaWxlczoNCmBgYHtyIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIGVjaG89RkFMU0V9DQojIGNoYW5nZSB3ZA0Kc2V0d2QoIkM6L1VzZXJzL2dhdmluL0JveC9NT0RJUyBEYXRhL0FwcEVFQVJTIERhdGEvY2g0LXYyMCIpDQojIGdldCBzaXRlIG5hbWVzIGZyb20gZGlyZWN0b3J5DQpzaXRlLm5hbWVzIDwtIGxpc3QuZmlsZXMocGF0dGVybiA9ICJcXC5jc3YkIixmdWxsLm5hbWVzPVRSVUUpDQpzaXRlLm5hbWVzDQogICMgY3JlYXRlIGxpc3Qgb2YgcHJvZHVjdHMNCm1vZGlzIDwtIGxpc3QoKQ0KIyBjcmVhdGUgYSBmdW5jdGlvbiB0byByZWFkIG1hbnkgY3N2cw0KcmVhZC5tYW55LmNzdiA8LSBmdW5jdGlvbihpKXsNCiAgcmVhZC5jc3YoaSwgaGVhZGVyPVRSVUUpDQp9DQojIGV4ZWN1dGUgZnVuY3Rpb24gd2l0aCBzaXRlLm5hbWVzDQptb2RpcyA8LSBsYXBwbHkoc2l0ZS5uYW1lcywgcmVhZC5tYW55LmNzdikNCm1vZGlzDQpgYGANClNpbXBsaWZ5IGRhdGEgYnkgcmVtb3ZpbmcgdW5uZWNzc2FyeSBmaWVsZHM6DQpgYGB7ciBlY2hvID0gRkFMU0V9DQp2aXMubmFtZXMgPC0gbmFtZXMobW9kaXNbWzFdXVtjKDMsNCw1LDksMTAsMTQpXSkgICMwOUExIHN1cmZhY2UgcmVmIGIwMiBiMDUNCmxzdC5uYW1lcyA8LSBuYW1lcyhtb2Rpc1tbMl1dW2MoMyw0LDUsOSwxMCwxNSwyNCldKSAjIzExQTIgTFNURE4NCmV2aS5uYW1lcyA8LSBuYW1lcyhtb2Rpc1tbM11dW2MoMyw0LDUsOSwxMyldKSAjIzEzUTEgRVZJDQpsYWkubmFtZXMgPC0gbmFtZXMobW9kaXNbWzRdXVtjKDMsNCw1LDEwLDE1KV0pICMjIDVBMkgNCm5hbWVzLnZlYyA8LSBsaXN0KHZpcy5uYW1lcyxsc3QubmFtZXMsIGV2aS5uYW1lcywgbGFpLm5hbWVzKQ0KDQptb2Rpcy5zaW1wbGUgPC0gbGlzdCgpDQpmb3IgKGkgaW4gMTpsZW5ndGgobW9kaXMpKXsNCiAgICAgbW9kaXMuc2ltcGxlW1tpXV0gPC0gYXNfdGliYmxlKG1vZGlzW1tpXV1bbmFtZXMudmVjW1tpXV1dKQ0KICAgICBtb2Rpcy5zaW1wbGVbW2ldXSREYXRlIDwtIGFzX2RhdGUobW9kaXMuc2ltcGxlW1tpXV0kRGF0ZSkNCn0NCiNzcGxpdCBuaWdodCBhbmQgZGF5IExTVDsgZXZpIGFuZCBORFZJOyBMQUkgYW5kIGZQQVIuDQptb2Rpcy5zaW1wbGVbWzVdXSA8LSBtb2Rpcy5zaW1wbGVbWzJdXSAlPiUgc2VsZWN0KExhdGl0dWRlLCBMb25naXR1ZGUsIERhdGUsIE1PRDExQTJfMDA2X0xTVF9OaWdodF8xa20sIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBNT0QxMUEyXzAwNl9RQ19OaWdodF9NT0RMQU5EX0Rlc2NyaXB0aW9uKQ0KbW9kaXMuc2ltcGxlW1syXV0gPC0gbW9kaXMuc2ltcGxlW1syXV0gJT4lIHNlbGVjdChMYXRpdHVkZSwgTG9uZ2l0dWRlLCBEYXRlLCBNT0QxMUEyXzAwNl9MU1RfRGF5XzFrbSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgTU9EMTFBMl8wMDZfUUNfRGF5X01PRExBTkRfRGVzY3JpcHRpb24pDQpuYW1lcyhtb2Rpcy5zaW1wbGUpIDwtIGMoInZpcyIsImxzdF9kYXkiLCJldmkiLCJsYWkiLCJsc3RfbmlnaHQiKQ0KbW9kaXMuc2ltcGxlDQpgYGANCkxvb2sgYXQgd2hhdCB0aGUgZGF0YSBxdWFsaXR5IGxldmVscyBhcmUsIGNvbnZlcnQgZ29vZCB0byAxLCBrZWVwIG9ubHkgMXMNCmBgYHtyIGVjaG89IEZBTFNFfQ0KI2NvbnZlcnQgbGV2ZWxzIHRvIGZhY3Rvcg0KbGV2ZWxzKG1vZGlzLnNpbXBsZVtbMV1dW1s2XV0pIDwtIGFzLmZhY3RvcihjKDAsMSwwKSkNCmxldmVscyhtb2Rpcy5zaW1wbGVbWzJdXVtbNV1dKSA8LSBhcy5mYWN0b3IoYygwLDEsMCkpDQpsZXZlbHMobW9kaXMuc2ltcGxlW1szXV1bWzVdXSkgPC0gYXMuZmFjdG9yKGMoMCwwLDEsMCkpDQpsZXZlbHMobW9kaXMuc2ltcGxlW1s0XV1bWzVdXSkgPC0gYXMuZmFjdG9yKGMoMSwwKSkNCmxldmVscyhtb2Rpcy5zaW1wbGVbWzVdXVtbNV1dKSA8LSBhcy5mYWN0b3IoYygwLDEsMCkpDQoNCiNyZW5hbWUgdmFyaWFibGVzDQpuYW1lcyhtb2Rpcy5zaW1wbGVbWzFdXSkgPC0gYygiTEFUIiwgIkxPTkciLCAiREFURSIsICJiMDIiLCAiYjA1IiwgIlFVQUxJVFkiKQ0KbmFtZXMobW9kaXMuc2ltcGxlW1syXV0pIDwtIGMoIkxBVCIsICJMT05HIiwgIkRBVEUiLCAiTFNURCIsICJRVUFMSVRZIikNCm5hbWVzKG1vZGlzLnNpbXBsZVtbM11dKSA8LSBjKCJMQVQiLCAiTE9ORyIsICJEQVRFIiwgIkVWSSIsICJRVUFMSVRZIikNCm5hbWVzKG1vZGlzLnNpbXBsZVtbNF1dKSA8LSBjKCJMQVQiLCAiTE9ORyIsICJEQVRFIiwgIkxBSSIsICJRVUFMSVRZIikNCm5hbWVzKG1vZGlzLnNpbXBsZVtbNV1dKSA8LSBjKCJMQVQiLCAiTE9ORyIsICJEQVRFIiwgIkxTVE4iLCAiUVVBTElUWSIpDQoNCiMgbW9kaXMuc2ltcGxlDQptb2Rpcy5maWx0ZXJlZCA8LSBsaXN0KCkNCmZvciAoaSBpbiAxOmxlbmd0aChtb2Rpcy5zaW1wbGUpKXsNCiAgbW9kaXMuZmlsdGVyZWRbW2ldXSA8LSBhc190aWJibGUobW9kaXMuc2ltcGxlW1tpXV0gJT4lIGZpbHRlcihRVUFMSVRZID09IDEpKQ0KfQ0KbW9kaXMuZmlsdGVyZWQNCm5hbWVzKG1vZGlzLmZpbHRlcmVkKSA8LSBjKCJ2aXMiLCJsc3RfZGF5IiwiZXZpIiwibGFpIiwibHN0X25pZ2h0IikNCmBgYA0KTWFrZSBhIHBsb3Qgb2YgZGF5dGltZSBMU1QgdG8gbWFrZSBzdXJlIHRoaW5ncyBsb29rIE9LDQpgYGB7cn0NCmdncGxvdChtb2Rpcy5maWx0ZXJlZFtbMl1dLCBhZXMoREFURSwgTFNURCwgY29sID0gYXMuZmFjdG9yKExBVCkpKSArDQogIGdlb21fcG9pbnQoKQ0KYGBgDQpDcmVhdGUgYSBkYXRhZnJhbWUgZm9yIGVhY2ggcHJvZHVjdCBjb25zaXN0aW5nIG9mIExBVCwgTE9ORywgREFURSBhbmQgdGhlIHByb2R1Y3QgZmllbGQgKGUuZy4gTFNURCkNCmBgYHtyfQ0KbHN0X2QuZmlsdGVyZWQgPC0gbW9kaXMuZmlsdGVyZWQkbHN0X2RheSAlPiUgbXV0YXRlKExTVEQgPSBMU1REKSAlPiUgc2VsZWN0KExBVCwgTE9ORywgREFURSwgTFNURCkNCmxzdF9uLmZpbHRlcmVkIDwtIG1vZGlzLmZpbHRlcmVkJGxzdF9uaWdodCAlPiUgbXV0YXRlKExTVE4gPSBMU1ROKSAlPiUgc2VsZWN0KExBVCwgTE9ORywgREFURSwgTFNUTikgDQpldmkuZmlsdGVyZWQgPC0gbW9kaXMuZmlsdGVyZWQkZXZpICU+JSBtdXRhdGUoRVZJID0gRVZJKSAlPiUgc2VsZWN0KExBVCwgTE9ORywgREFURSwgRVZJKSANCmIwMi5maWx0ZXJlZCA8LSBtb2Rpcy5maWx0ZXJlZCR2aXMgJT4lIG11dGF0ZShiMDIgPSBiMDIpICU+JSBzZWxlY3QoTEFULCBMT05HLCBEQVRFLCBiMDIpDQpiMDUuZmlsdGVyZWQgPC0gbW9kaXMuZmlsdGVyZWQkdmlzICU+JSBtdXRhdGUoYjA1ID0gYjA1KSAlPiUgc2VsZWN0KExBVCwgTE9ORywgREFURSwgYjA1KQ0KbGFpLmZpbHRlcmVkIDwtIG1vZGlzLmZpbHRlcmVkJGxhaSAgJT4lIG11dGF0ZShMQUkgPSBMQUkpICU+JSBzZWxlY3QoTEFULCBMT05HLCBEQVRFLCBMQUkpDQoNCmBgYA0KQ3JlYXRlIGEgZnVsbCBkYXRhZnJhbWUgd2l0aCBlYWNoIHNpdGUgbmFtZS9sYXRsb25nIHJlcGVhdGVkIG4gPSBsZW5ndGgoMjAwMS0wMS0wMSB0byAyMDE0LTEyLTMxKSAoRkxVWENPTSBkYXRhIHdpbmRvdykuDQpUaGVuIHJpZ2h0X2pvaW4gZWFjaCBtb2RpcyBwcm9kdWN0IHRvIHRoZSBzaXRlIG5hbWUvYWxsIGRhdGVzIHRpYmJsZSB0byBwcm9kdWNlIGEgY29tcGxldGUgZGF0YWZyYW1lIG9mIG1vZGlzIGRhdGEuDQpgYGB7cn0NCmZ1bGwuZGF0ZSA8LSBhc190aWJibGUocmVwKGFzX2RhdGUoYyhhc19kYXRlKCIyMDAxLTAxLTAxIik6YXNfZGF0ZSgiMjAxNC0xMi0zMSIpKSksNTkpKSAjIyByZXAgYnkgbnVtYmVyIG9mIHNpdGVzIA0KbmFtZXMoZnVsbC5kYXRlKSA8LSAiREFURSINCmZ1bGwuZGF0ZSA8LSBmdWxsLmRhdGUgJT4lIGFkZF9jb2x1bW4oSUQgPSByZXAoYmFtcy5zaXRlcyRJRCwgZWFjaCA9IGxlbmd0aChmdWxsLmRhdGUkREFURSkvNTkpKSAlPiUgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhZGRfY29sdW1uKExBVCA9IHJlcChiYW1zLnNpdGVzJExBVCwgZWFjaCA9IGxlbmd0aChmdWxsLmRhdGUkREFURSkvNTkpKSAlPiUNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYWRkX2NvbHVtbihMT05HID0gcmVwKGJhbXMuc2l0ZXMkTE9ORywgZWFjaCA9IGxlbmd0aChmdWxsLmRhdGUkREFURSkvNTkpKQ0KDQptb2Rpcy5mdWxsIDwtIGZ1bGxfam9pbihmdWxsLmRhdGUsIGxzdF9kLmZpbHRlcmVkLCBieSA9YygiREFURSIsIkxBVCIsIkxPTkciKSkNCm1vZGlzLmZ1bGwgPC0gZnVsbF9qb2luKG1vZGlzLmZ1bGwsIGxzdF9uLmZpbHRlcmVkLCBieT1jKCJEQVRFIiwiTEFUIiwiTE9ORyIpKQ0KbW9kaXMuZnVsbCA8LSBmdWxsX2pvaW4obW9kaXMuZnVsbCwgZXZpLmZpbHRlcmVkLCBieT1jKCJEQVRFIiwiTEFUIiwiTE9ORyIpKQ0KbW9kaXMuZnVsbCA8LSBmdWxsX2pvaW4obW9kaXMuZnVsbCwgYjAyLmZpbHRlcmVkLCBieSA9IGMoIkRBVEUiLCdMQVQnLCJMT05HIikpDQptb2Rpcy5mdWxsIDwtIGZ1bGxfam9pbihtb2Rpcy5mdWxsLCBiMDUuZmlsdGVyZWQsIGJ5ID0gYygiREFURSIsIkxBVCIsIkxPTkciKSkNCm1vZGlzLmZ1bGwgPC0gZnVsbF9qb2luKG1vZGlzLmZ1bGwsIGxhaS5maWx0ZXJlZCwgYnk9YygiREFURSIsIkxBVCIsIkxPTkciKSkNCg0KbW9kaXMuZnVsbCA8LSBtb2Rpcy5mdWxsICU+JSBkaXN0aW5jdChJRCwgREFURSwgLmtlZXBfYWxsID0gVFJVRSkNCnN0cihtb2Rpcy5mdWxsKQ0KYGBgDQoNCkZpbmlzaGVkISBTYXZlIGFzIC5jc3YNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9DQojIHNldCB3ZA0Kc2V0d2QoIkM6L1VzZXJzL2dhdmluL0JveC9NT0RJUyBEYXRhL0FwcEVFQVJTIERhdGEiKQ0Kd3JpdGUuY3N2KG1vZGlzLmZ1bGwsICJWMi4wX01PRElTLmNzdiIpDQpgYGANCg0KDQoNCg0K